On Approximating the Stationary Distribution of Time-reversible Markov Chains
نویسندگان
چکیده
Approximating the stationary probability of a state in a Markov chain through Markov chain Monte Carlo techniques is, in general, inefficient. Standard random walk approaches require Õ(τ/π(v)) operations to approximate the probability π(v) of a state v in a chain with mixing time τ , and even the best available techniques still have complexity Õ(τ/π(v)); and since these complexities depend inversely on π(v), they can grow beyond any bound in the size of the chain or in its mixing time. In this paper we show that, for time-reversible Markov chains, there exists a simple randomized approximation algorithm that breaks this “small-π(v) barrier”.
منابع مشابه
On Non-reversible Markov Chains
Reversibility is a suucient but not necessary condition for Markov chains for use in Markov chain Monte Carlo simulation. It is necessary to select a Markov chain that has a pre-speciied distribution as its unique stationary distribution. There are many Markov chains that have such property. We give guidelines on how to rank them based on the asymptotic variance of the estimates they produce. T...
متن کاملInequalities for Rare Events in Time-reversible Markov Chains Ii
A previous paper discussed explicit bounds in the exponential approximation for the distribution of the waiting time until a stationary reversible Markov chain rst enters a \rare" subset of states. In this paper Stein's method is used to get explicit (but complicated) bounds on the Poisson approximation for the number of non-adjacent visits to a rare subset.
متن کاملA Modification of Neal’s Algorithm for a Continuous State Space and an Application to the Fokker-planck Equation
The Metropolis-Hastings algorithm generates correlated samples from a target distribution by constructing a Markov chain which has as its stationary distribution the desired target distribution. One property of this algorithm is that it creates reversible Markov chains. As a result, reversible chains are often used in Monte Carlo simulations. Reversible Markov chains also have the added benefit...
متن کاملInequalities for Rare Events in Time-reversible Markov Chains I
The distribution of waiting time until a rare event is often approximated by the exponential distribution. In the context of rst hitting times for stationary reversible chains, the error has a simple explicit bound involving only the mean waiting time ET and the relaxation time of the chain. We recall general upper and lower bounds on ET and then discuss improvements available in the case ET wh...
متن کاملHitting and Convergence Time , and Flow Rate , Parameters for Reversible Markov
The elementary theory of general nite Markov chains (cf. Chapter 2) focuses on exact formulas and limit theorems. My view is that, to the extent there is any intermediate-level mathematical theory of reversible chains, it is a theory of inequalities. Some of these were already seen in Chapter 3. This chapter is my attempt to impose some order on the subject of inequalities. We will study the fo...
متن کامل